DOCUMENT RESUME 



ED 255 635 



CE 040 861 



AUTHOR 

TITLE ^ 

INSTITUTION • 

SPONS AGENCY 

PUB DATE 
CONTRACT 
NOTE 

AVAIuABLE FROM 



PUB TYPE 

EDRS PRICE 
DESCRIPTORS 



Desy, Jeanne; And Others 

Assessing Learning Time at the Co-op Training 
Station. Special Publication Series No. 50. 
Ohio State Univ., Columbus. National Center for 
Research in Vocational Education. 
Office of Vocational and Adult Education (ED), 
Washington, DC. 
85 

300-83-0016 

66p.; For related documents, see CE 040 899-900. 
National Center Publications, Box F, National Center 
for Research in Vocational Education, 1960 Kenny 
Road, Columbus, OH 43210-1090 (SN50— $6.25) . 
Guides - Non-Classroom Use (055) 

MF01/PC03 Plus Postage. 

Check Lists; *Cooperative Education; Evaluation 
Criteria; Evaluation Methods; Guidelines; *0n the Job 
Training; Postsecondary Education; Program 
Evaluation; Records (Forms); Research Design; 
Research Methodology; ^Research Utilization; 
Secondary Education; *Time Factors (Learning); *Time 
Management; Time on Task; ^Vocational Education; 
Worksheets 

ABSTRACT 

This handbook is intended to guide cooperative 
education coordinators, teachers, inservice coordinators, and teacher 
educators in conducting time-use analyses. Chapter 1, an introduction 
to the guide, discusses the issue of evaluation in cooperative 
education, the benefits of timl-use analyses, and common objections 
to time-use assessment. Examined in the second chapter are various 
applications of time-use analyses, including developing and modifying 
training plans, building time consciousness, targeting priorities, 
fostering detachment, increasir^g efficiency of placement at traininr^ 
stations, solving problems, forming and substantiating useful 
evaluations, and freeing up time. Chapter 3 deals with the phases 
conducting a time-use study. Various aspects of using the results 
time-use studies are discussed in chapter 4, including gathering 
additional information, evaluating the learning experience, 
identifying problem areas, modifying the training plan, and talking 
over the results. Two observation forms and five worksheets for use 
in conducting time-use analyses are appended, and numerous sample 
forms and checklists are included throughout the text of the 
handbook. (MN) 
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FOREWORD 



Cooperative vocational education is a nnethod for blending employment and education in 
order to strengthen the preparation of youth for entry into the labor market. Specific information 
on students' training station experience has not always been available to cooperative education 
coordinators. For example, data are often lacking on the amount of time cooperative education 
students spend in training station activities that are defined in the training plan and that contribute 
to the acquisition of work/^RHIs and knowledge. Such information is vital for those individuals who 
are interested in evalualmg a ndHqn proving cooperative vocational education programs. 

This handbook/is intended to guide cooperative education coordinators, teachers, inservice 
coordinators, and4eacher educators in conducting time-use analyses. It can be used in both 
secondary and postsecondary training station settings for various purposes, such as process evalu- 
ation for program improvement, research for explanatory investigations, and inservice programs 
for cooperative education coordinators or teachers. 

Detailed step-by-step procedures involved in planning and conducting time-use observation 
and analyzing and interpreting data are presented in this handbook. Emphasis is placed on using 
the results to work toward quality cooperative vocational education programs. 

This handbook was developed in the Evaluation and Policy Division of the National Center for 
Research in Vocational Education under a contract with the Office of Vocational and Adult Educa- 
tion. U.S. Department of Education. We wish to thank N. L. McCasliri, Associate Director, and 
Stephen J. Franchak, Project Director. The major responsibility for the whting of this document 
was undertaken by Jeanne Desy, Program Associate. Appreciation is also extended to Lee Norton. 
Graduate Research Associate, and Stephen J. Franchak for major assistance in the preparation of 
the handbook. 

A special note of appreciation is given for the many hours of typing done by Marjorie Arnold, 
Cheryl Trayser. and Sharyn Eberhart. Graphic services were directed by Ernie Spaeth, and the 
design work was completed by Dennis Mathias. Appreciation is extended to Janet Kiplinger for her 
editorial assistance. We are also grateful to Roy L. Butler, Senior Research Specialist, and Michael 
R. Crowe. Research Specialist, from the National Center for Research in Vocational Education, for 
their review and critique of the draft copy. 



Robert E. Taylor 
Executive Director 
National Center for Research 
in Vocational Education 
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EXECUTIVE SUMMARY 



One of the recommendations in the report A Nation at Risk (National Commission on Excel- 
lence in Education 1983) was to increase student time for learning. The assumption that more effi- 
cient time use can increase students' achievement has been verified in a number of research 
studies, furthermore, time use is one variable that employers, training sponsors, and cooperative 
education coordinators can control to increase the quality of the student's learning experience. 
Better use of time at the training station can be expected to lead to an increase in the vocational 
student's learning productivity. The cooperative education coordinator and training sponsor can 
"teach smarter" and the student can "work smarter" by changing the ways time is managed at the 
training station. 

The procedures presented in this handbook are based primarily on a study of secondary 
cooperative education students at selected training stations (Franchak, Norton, and Desy 1985). 
Additional information was obtained from studies of time use in secondary and postsecondary 
vocational education classes (Halasz and Behm 1983; Halasz. Behm, and Fisch 1984). 

This handbook is a companion volume to the publication Managing Learning Time: A Voca- 
tional Educator's Handbook (Halasz and Desy 1984). That handbook is a guide for teachers, 
administrators, supervisors, inservice coordinators, and teacher educators in conducting time-use 
analysis in the vocational education classroom. Time-use analysis at the training station is similar 
in a number of ways; however, the differences between the classroom setting and the work site 
dictate the use of alternative procedures. This handbook addresses time-use assessment at the 
triiiining station. It provides the cooperative education coordinator with procedures designed spe- 
cifically to be used in conducting time-use analyses of cooperative education students at the train- 
ing station. The three stages of cooperative education time-use analysis are as follows: 

1. Examining the benefits of time-use analysis 

2. Conducting a time-use study 

3. Working with the results 

The handbook is divided into four chapters. Chapter 1 serves as an introduction to the hand- 
book. Chapter 2 discusses the applications of time-use analysis. Chapter 3 comprises the steps 
required in designing, planning, and conducting time-use analyses at the training station. Chapter 
4 presents specific strategies to increase time on task at the training station. Moreover, this chap- 
ter explains how to apply time-use assessment results to improve the learning experience of indi- 
vidual students and to effect overall program improvement. These chapters are arranged in the typ 
cal chronology of completing a time-use study. However, a number of steps are interrelated, and, 
in actual practice, concepts from several chapters may be used simultaneously. 
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Each chapter in the handbook is self-contained. The reader can expect to become familiar 
with major concepts, alternative approaches (where applicable), and specific techniques associ- 
ated with the topic of each chapter. Within each chapter are references for further information. 
Examples are given throughout to illustrate the yse of worksheets, observation forms, and 
checklists. 

Overall, this handbook is designed to help cooperative education coordinators manage their 
programs more effectively. Time-use assessment is presented as a valuable means of obtaining 
information for the evaluation and improvement of cooperative education programs. 
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CHAPTER 1 



INTRODUCTION 



Evaluation In Cooperative Education 

There are many purposes cf educational program evaluation. Often, one central purpose is to 
determine how well program objectives are being met. This purpose is not always obvious in the 
informal evaluations of cooperative education programs that are shared when administrators 
gather at conferences and workshops. Programs are discussed in terms of how many students are 
placed at training stations, how satisfied employers are, how many students are hired permanently 
atter graduation, and so on. Although there is sometimes pressure to meet these objectives, they 
are seldom the major objectives of the cooperative education program. Cooperative education is 
intended to be. as the term implies, educational. The training station is meant to help individuals 
prepare for employment by providing training. Therefore, the amount of learning relating to tech- 
nical skills, basic skills, and employability skills at the training stations is the appropriate subject of 
evaluation. 

Because other concerns intrude, the learning objective sometimes assumes a secondary role. 
Moreover, it is not easy to measure learning at a training station. The two obvious sources of 
information, the student and the employer, may be vague and general in their statements. They 
may overstate or understate the learning that is taking place. The employers may be very pleased 
with the students' proficiency and uniformly positive in their evaluations, but not have time to 
observe that the student's learning (relating to technical skills practice, basic skills, and employ- 
ability skills) could be enhanced through more effective use of time or better supervision methods. 

An evaluation is no better than the information upon which it is based. One effective way to 
gather information about a cooperative education program is through systematic observation of 
training stations. The information gathered should be as objective as possible, taking note of the 
specific tasks in which the student is engaged. Time-use analysis, coupled with other information 
gained through direct observation, is a procedure for gathering this kind of information for the 
evaluation of individual training stations and educational programs. 



Timefuse Assessment 

Fundamentally, time-use assessment involves recording what a student does and then calcu- 
lating the percentage of time spent on task and off task. At this point, the time-use analysis has pro- 
duced data that can assist in making informed decisions about the learning experience. This is the 
purpose of time-use assessment: to facilitate effective decision making. It can do so by providing a 
useful, objective base of information. 

On-task activities are those related, directly or indirectly, to the learning activities (tasks) in 
which the student will engage, as defined by the training plan between the training sponsor and 
the cooperative education supervisor. Time on task is the amount of time spent on those learning 
activities. 

1 
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Time-use assessment usually measures more than overall time spent on learning activities. 
This method also makes it possible to look carefully at the kinds of skills a student uses at the 
training station, including basic, technical, and human interaction skills. It can also be used to 
measure what is happening when the student is o^^ task, either waiting for directions, socializing, 
or taking a scheduled break. Figure 1 presents a breakout of factors or how students may use time 



The idea that time spent on learning tasks is directly related to achievement in the task area 
was proposed by John Carroll in 1963. Since then, the relationship of time on task to learning has 
been the subject of numerous studies. Bloom (1974). Fisher et al. (1978), and Wiley and Harnisch- 
feger (1974) all found that students learned more when they spent more time on task. 

Time-on-task research also indicates that learning time must be not only used, but used well 
(Halasz, Behm, and F.'sch 1984: Siedentop 1983). Stallings (1980) found that merely increasing 
engaged time does not necessarily produce more learning; achievement depends on how time is 
used. Schneider (1980) pointed out that it is important to consider only the actual amount of time 
spent performing a task, since the total time taken can include nontask activities, such as waiting 
for materials or instructions. These relate directly to the cooperative education coordinator's find- 
ings about student time on task at the work site. 

Ideally, students will be on task most of the time, performing tasks that increase their skills 
and knowledge. Hilgard and Bower (1975) state that "to learn means to gain knowledge through 
experience" (p. 2). Learning can be defined as the relatively p.^rmanent change in a student's 
knowledge or behavior due to experience (Mayer 1982, p. 1,05i^). Simply working, however, does 
not guarantee learning. Moreover, the optimum times needed for learning specific skills is not 
known, & d how long a student needs to work at a skill in order to master it is still a matter for 
individual decision. 



The concept of measuring students* time use is faniiliar to most educators today— although 
the process itself is less well understood, and may seem ambiguous and complex. The method pre- 
sented here is new to cooperative vocational education, having been developed through a recent 
National Center study on assessing cooperative education students' time use at the training station 
(Franchak, Norton, and Desy 1985). Cooperative education supervisors will find such assessment 
familiar, having made unstructured assessments while visiting training stations, watching students 
at work, and noting how much students seem to be working and learning. Such visits are com- 
monly used to evaluate the quality of the cooperative education learning experience. Experiencer- 
supervisors can rightly claim that these assessments of the training station and the student are 
very useful. 

Systematic observation of time use has the same purpose as informal observation but differs 
in several ways. The most important is that time-use observation results in quantified information. 
Timefuse analysis is a procedure for observing students' activities at the training station as objec- 
tively as possible and recording these activities. 

The results of the systematic method are sometimes very surprising. Retail students placed at 
a cash register in the hosiery department of a large department store may complain that they 



at the training station. 



Time Use and Learning 



Time Use at the Training Station 
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AT COOPERATIVE EDUCATION 
TRAINING STATIONS 




Figure 1. Factors of student time use at cooperative education training stations 



spend a lol of time "doing nothing." But a time-use analysis may show that only 20 per^cenf of the 
students' time is "down" time or. alternatively, that 80 percent of the time the students are idle, 
waiting for customers. The results will determine how the cooperative education supervisor and 
training sponsor approach the problem of the students' boredom. The precision of these data is 
the key difference between time-use evaluation and less formal evaluation. 

Until recently, time-use assessment was focused on the classroom, where on-task activities 
were defined as those related, directly or indirectly, to the purpose of the class. Directly related 
tasks were defined as those in which the student was actually engaged in learning, such as work- 
ing on a word processor or listening to a lecture on safety practices. Indirectly related tasks, such 
as sweeping the work area before leaving class, might not have added to learning, but were con- 
sidered a necessary part of the learning task. 

At the training station, time-use analysis may at first seem radically different from classroom 
time-use analysis, but it is not. The student is now being paid to do a job. but the cooperative edu- 
cation job is designed to be more than a job; it is a learning experience. Time-use analysis is 
designed to measure that learning experience. 



Common Objection)! to Time-Use Assessment 

Like all new ideas, the concept of measuring time use has met with some opposition. The ^ 
method can seem formidably statistical, and the rewards may not be obvious at first glance. In the 
course of developing the procedures given here, the staff at the National Center for Research in 
Vocational Education met with many cooperative education coordinators. During these meetings, 
certain objections were brought up time and again. These objections are natural responses to this 
new method of gathering information. Our replies, in turn, are based on our experience with time- 
use analysis and reflect our belief in the usefulness of this method. Following are the concerns we 
heard expressed and our replies. 

"/ don't have time. " Very few cooperative education coordinators have time to spare. 

However, although time on task is primarily designed to lead to bet- 
ter informed, more consistent evaluations, it can save time as well. 
More information about training stations should lead to better deci- 
sions in placing students in cooperative education jobs. Knowledge 
about the student's interactions at the work site helps in finding 
solutions to any problems that may arise. In general, good informa- 
tion is necessary for good management, and good management 
saves time. 

"My employers wouldn't That's always possible, but the key is to give the employer or train- 
like it. " ing sponsor the chance to say "Yes" or "No." Setting up the obser- 

vation in advance is helpful; chapter 2 talks about how to explain 
what you will be doing. 

The employer stands to benefit from your increased knowledge. For 
example, you might see that employers in one program area are ask- 
ing students to perform tasks for which they really are not prepared. 
If so. the necessary skills can be worked into the curriculum, so that 
future students— and job applicants from the vocational education 
program— are more productive. 
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In working with time-use analysis of cooperative education stu- 
dents, we have asked many employers for the right to observe on 
site. Few have refused. 



"It's too^complicated. I 
don't like working with 
numbers," 



"I can't spend that much 
time on one student." 



'7 really can't do this for 
all my students/' 



"You can't learn much in 
one visit. " 



"People act differently 
when you're watching/ 



Probably because the observation form is detailed, many people do 
see this method as complex. Our observers have found, however, 
that once they completed their first observation the procedure was 
easy to follow. This handbook is devoted to making time-use analy- 
sis as simple and straightforward as possible. 

Ideally, a time-use analysis involves more than one visit to a training 
station. Perhaps several observations at monthly intervals can be 
made. Most cooperative education coordinators do visit their train- 
ing stations periodically, although they may not observe formally. It 
is worth noting that observation gives information on more than^the 
student's learning experience. It provides data about the exact skills 
needed in this position and about the nature of this workplace, 
information that wilt be useful in program management long after 
the student has graduated. 

That may be true, but it is still worthwhile to do it for some. An 
alternative is to limit observation of each student; even 1 hour at a 
training station can yield useful information. 

It's true that information from one short session may not provide a 
complete database. It's better to return at different times of the 
workday and different days of the week. But the truth is that 
extended observation is impractical for many coordinators, even 
when the help of vocational teachers is enlisted. Nonetheless, some 
information is better than none. The important thing to remember 
when evaluating the results of a single visit is that your information 
is incomplete and should not be taken as the last word on that stu- 
dent's learning experience at that training station. 

Observers should be unobtrusive and as far from the student as 
possible. Even so, the presence of an observer is naturally felt. More 
often than not, students and others at the work site are on their best 
behavior at first, like anyone who is conscious of being watched. 
Especially during the first observation of a site, results may err on 
the positive side, and evaluation should be tempered by that 
knowledge. 

However, the processes at that training station should remain essen- 
tially the same, in terms of the tasks the student performs. The stu- 
dent can work more diligently or carefully than usual, but probably 
cannot do other than the usual work. As a result, evaluation of the 
learning experience, the purpose of the observation, is seldom sig- 
nificantly impeded by the. fact that students know they are being 
observed. 
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'7 don't need this— I have Time-use analysis is not presented here as the one-and-only way of 
my own ways of gathering irtformation and evaluating the quality of a student's edu- 

evaluating, " cation at a iraining station. It does have a notable advantage over 

other methods, however: it is relatively objective. Time-use analysis 
can be us^d to enrich the information base for any method of 
evaluatio^. 



Summary 

Figure 2 on the following page ghes some of the central statements from chapter 1 . As these 
statements emphasize, time-use assessment can provide information for program evaluation and 
improvement. In addition, this method supplies information that can be used for more efficient and 
effective program management. 
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The amount of learning at the training 
stations in the program is an appro- 
priate subject of evaluation. 



• Learning time must be not only used, 
but used well. 

• Good information is necessary for 
good management, and good 
management saves time. 

• The purpose of time-use assessment is 
to facilitate effective decision making. 

• Time-use analysis can be used to 
enrich the information base for any 
method of evaluation. 



Figure 2. Ir.iroduction— key concepts 
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CHAPTER 2 

EXAMINING THE BENEFITS OF TIME-USE ANALYSIS 



Since time-use analysis requires learning a new procedure clearly, the benefits must make the 
effort worthwhile. This chapter, therefore, presents beneficial uses for time-use assessment in a 
cooperative education program. 



Developing and Modifying Training Plans 

In formulating training plans, it is best to specify not^only the tasks the student will perform 
but also the percentage of time the student will spend on various tasks. Observations of students' 
time use at training stations provide a standard that can be applied to other students in the same 
vocational service area. 

As individual students are observed, the information from the observation can be used to mod- 
ify their training plans. You may find, for example, that a student spent 70 percent of the observa- 
tion time performing a routine task, such as taking phone messages. This information may lead 
you to discuss modifying the training plan with the employer, so that the student will spend 50 
percent of the tr^Jning station time learning new skills by working on a word processor. 



Dullding Time Conscloutness 

When studentsVtlme is logged, they become more conscious of how they spend their time. 
Often, they begin to notice how much time they spend off task and how much of their on-task time 
represents learning. Your evaluation sessions with individual students will tend to reinforce the 
premise that the training station is to be a learning experience and that students should do what 
they can to maximize their own proficiency. 

Employers can also learn from the observation. Beyond seeing how the student spends time at 
the training station, the employer can gain a better understanding of the relationship between the 
way the student's time is allocated and the quality of the student's experience. 



Targeting Priorities 

At a typical work site, hundreds of details vie for attention. It is impossible to note everything 
and all too easy to notice just one or two events of a certain kind. Designing or choosing an obser- 
vation form forces you to decide what matters enough to be recorded. For instance, the forms 
used in the study conducted by Franchak and Norton (1985) includes employability skills. The 
student learning an appropriate attitude toward work; gaining knowledge of the world of work: or 
making efforts toward seeking, maintaining, or advancing in a job. These items were targeted on 
the form, because we believe learning employability skills is an important part of the cooperative 
education experience. 

9 
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Fostering Detachment 

One of the greatest challenges in evaluation is establishing the detachment necessary to fair 
judgment. Detachment is especially difficult when emotions are involved; educators who care 
deeply about their students must take pains to evaluate objectively. Systematic observation 
employing time-use assessment deals with facts, not feelings or impressions. 



Increasing Efficiency of Placement at 
Training Stations 

Although gathering information about the student's learning experience is the primary goal of 
observation, the observer also learns a great deal about the training station and the demands and 
opportunities of the job. This knowledge can be used in making decisions about placing students 
at sites for which they must have the necessary range of skills to work successfully and where they 
can learn from the experience. It can also forestall some problems of human Interaction and per- 
sonality conflicts. Additionally, observation data can suggest good placements; for instance, a 
hard-working student who demands step-by-step supervision might learn from working at a site 
where the supervisor is helpful but encourages the student to work independently. Knowledge 
about a situation confers the power to make more effective decisions. 



Solving Problems 

Some of the problems cooperative education coordinators encounter are unique and unfore- 
seeable, although many problems can be prevented by better informed cooperative education 
placements. All problems are more easily dealt with when a good Information base is available. 
The data gathered during observation can shed light on a student's complaint about obsolete or 
malfunctioning equipment. Observation of an inefficient work process at a training station may 
suggest that the employer is correct In saying the student spends too much time standing idle— 
but that this Is not the student's fault. Even personality problems, among the most difficult prob- 
lems to resolve, can be handled more easily when the Interactions at the site have been observed. 



Forming Useful Evfiluatlons 

The most productive evaluations are those that assess performance in realistic terms, offering 
positive reinforcement for work done well and specific suggestions for improvement. Time-use 
analysis makes possible the evaluation of specific aspects of the student's work and of the learning 
experience at the training station. 



Substantiating Evaluations 

Students and training sponsors see their own behavior subjectively. In communicating your 
evaluation to them, it is very helpful to have data on hand. For example, rather than telling an 
employer that the training station Is generally good, you might point out that the student is getting 
the opportunity to learn entrepreneurial skills. The notes on your observation form should show 
what these skills are and how much time the student spent on tasks in this area. Observation data 
also help students see their behavior more objectively. Students may feel they are working well 
until the results of analysis are presented, showing that they are off task socializing over 40 per- 
cent of the time. The objective and definite nature of time-use data is perhaps most helpful in 
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cases where a student feels the evaluation is biased or the cooperative education coordinator's 
decision unfair. With this method of analysis, a coordinator can show the information used to 
make the decision. As a rule, evaluation is more meaningful and acceptable when it is given with 
reference to the actual behavior observed. 



Most cooperative education coordinators tell us there isn't enough time in the workday to do 
their jobs the way they should be done. When training stations are scattered all over a rural 
county, or an urban area, and the cooperative education coordinator handles a hundred students 
or more, time is always a problem. 

Although time-use analysis takes time. It can ultimately save time by providing enough infor- 
mation to make better decisions than in the past. Matching a student to the best possible training 
station should mean fewer problems to Investigate and resolve; evaluating students at the work site 
is an excellent way to catch problems before they become severe. Decision making, a large part of 
a coordinator's job. should flow more smoothly and quickly when a good information base has 
already been compiled. 



Although this handbook stresses assessing time use for evaluation purposes, time-use infor- 
mation can also be used in other ways. Some of the uses presented in this chapter are highlighted 
in figure 3. 



Freeing Up Time 



Summary 
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• Information from the observation can 
be used to modify students' training 
plans. 

• Students become conscious of how 
they spend their time. 

• The observer also learns a great deal 
about the training station and the 
demands and opportunities of the job. 

• All problems are more easily dealt with 
when a good information base is 
available. 

• Evaluation is more meaningful and 
acceptable when it is given with 
reference to the actual behavior 
observed. 



Figure 3. Examining the benefits of time-use analysis— key concepts. 
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CHAPTER 3 



CONDUCTING A TIME-USE STUDY 



Preparing 



Much of the effort in time-use observation takes place before the actual observation. This is 
especially true of the first observation, since preparing includes learning how to use the form, 
choosing and training any other observers, and/developing your system for choosing training 
stations to observe. Once done, these preparations will take much less effort in the future. As this 
section emphasizes, all preparation should focus on your objectives— your purpose in assessing 
student time use at the training station. Essential steps in conducting a timefuse study are pre- 
sented in figure 4. 



Studying The Training Plan 

The training plan defines the experience the cooperative vocational education student is sup- 
posed to obtain at the training station. It should list both the types of work to which the student 
will be exposed and the tasks the student will perform and be given the opportunity to master dur- 
ing the training experience. The training plan is, therefore, an important document in your study of 
the time use of cooperative education students. In conducting a time-use study, the observer must 
know what confjitutes on-task behavior. The list of tasks in the training plan defines the student's 
on-task activities. 



Choosing Training Stations to Observe 

There are many possible reasons for analyzing cooperative education student time use; your 
objective will indicate which training stations should be observed. For example, one reason for 
analyzing student time use is to develop standards for a good cooperative education experience. If 
this is your objective, you will want to observe training stations that you believe offer students a 
high-quality training experience. These would almost certainly be traii:ing stations at which you 
have placed many students in the past with positive results. 

A second reason for analyzing student time use is to obtain information that will help you 
manage your cooperative education program. If this is your objective, you might choose those 
training stations about which you have the least information. These would probably be training 
stations that are relatively new to your program. 

It may be that your objective is to evaluate the learning experiences of individual students at 
their training stations, as part of an effort to maintain or improve the quality of your program. This 
objective is probably best met by observing all students in the program. However, that may not be 
feasible or realistic, given constraints on your time and resources that are available for conducting 
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Figure 4. Steps In conducting a time-use study 
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the study. In that case, you may develop a sampling plan to gather training station time-use data, 
and then use these results to make inferences about all the training stations. 



According to Backstrom 3nd Hursh (1963, p. 42), a good sample— , 

• Provides ways to determine the number of training stations that 'should be sampled; 

• Specifies the char. (probability) that any training station will beJncluded in the sample: 

• Enables us to estimate he w much error results from observing a sample of training 
stations instead of observing all of them; and 

• Allows us to determine the degree of confidence that can be placed in total population 
estimates from the sample. 

In general, Backstrom and Hursh (1963) indicate two minimum requirements in sampling: 
(1) the sample must be large enough so that estimates about the characteristics of the population 
can be made with relative precision, and (2) the sample must include people who together are 
representative of the population (p. 25). 

Sampling procedures have been classified under various categories. The two categories that 
are most commonly referred to are probability and nonprobabillty sampling. Probability sampling 
involves the selection of elements for the sample by chance procedures and with known probabili- 
ties of selection. In nonprobabillty sampling, elements are not selected by chance procedures, and 
probabilities of selection are not known. 

Sample size is a major concern In designing a sampling plan. No absolute standards exist 
regarding the "best" size of a sample, although tables appear in the literature that can provide 
"guesti mates" of the appropriate sample size. However, there are some helpful hints to bear in 
mind when determining your sample size. Chatobar and Lad (1974) suggest the following: 

• The more similar the population, the smaller the sample can be, since everyone is theoret- 
ically alike. 

• The more time, personnel, and money available, the larger the sample can be. 

• The more categories by which the sample data are to be generalized (e.g.. age, gender, 
type of cooperative vocational education program), the larger the sample must be in order 
to ensure that there are enough observations in each group so that valid inferences may 
be made. 

• The more you want to be sure of the sample information, the larger the sample you will 
need. In other words, the less error you can permit in the result, the larger the sample you 
will need. 

The reader is referred to a publication prepared by Gold and Morris (1977) for a clearly defined 
procedure for selecting a sample size. 

As this overview suggests, sampling is a complex field, and a casually chosen sample is likely 
to have little value. A small sample might, for Instance, suggest that students at the training 
stations in your programs are getting no opportunity to use basic skills. This might drastically mis- 
state the truth; perhaps the sample does not include certain types of businesses in which some 
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students would use basic skills often. It is important then, to recognize the limitations of your sam- 
ple procedure. 



For an overview of student time use at your program's training stations, be certain the most 
important factors are fairly represented. Such a sample might include both established and new 
training stations, cooperative education students who are new to cooperative education and those 
who are experienced, male and female students, and other elements you believe to be important. 
Again, be cautious in your interpretation of the results. 

Our warnings about sampling are not meant to be discouraging, 'but only to point out the 
hazards involved. Time-use information is a valuable addition to any program evaluation, and a 
valid sampling plan can aid in obtaining useful information when it is not possible to observe all 
training stations. 

Many textbooks and journal articles are available on sampling, as is research you may consult 
for more procedural details on designing an appropriate sampling plan and selecting a sample. 
The following texts are suggested: 

Fritz-Gibbon, C. T., and Morris, L. L. How to Design a Program Evaluatior). Beverly Hills- Sage. 
1978. 

Gold. B., and Morris. W. Student Accountability Model (SAM). Operations Manual. Sacra- 
mento. CA: Chancellor's Office, California Community College, 1977. 

Scheaffer. R. L.: Mendenhall. W.; and Ott, L. Elementary Survey Sampling. Belmont. CA: 
Wadsworth Publishing Company, 1979. 

Sudman, S. Applied Sampling. New York: Academic Press, 1976. 



The Observation Porm 

Observation form 1 shown in figure 5 is arranged for manual calculation of results. Observa- 
tion form 1-A in the Appendix is very similar to observation form 1 but is arranged for keypunching 
for computer analysis of the data collected. The difference in the two forms is that the keypunch- 
ready form (1-A) includes variable numbers (1-8 under the identification codes, 10-13 under the 
time. 14-21 under time-on-task activities, and 22-29 under time-off-task activities) that would be 
the colunnn numbers on a data card or tape in which the data would be punched. In duplicating 
forms for observation, be sureto choose one form or the other, depending on whether you want to 
calculate the results manually or through the use of a computer. 

The observation form contains two categories: (1 ) on-task behavior and (2) off-task behavior. 
On-task behavior, as previously discussed, includes all activities performed by the student that are 
related to the proposed training experience outlined in the student's training plan. Off-task activi- 
ties may be either sanctioned or nonsanctioned. An example of a sanctioned, off-task activity is ^ 
the scheduled break, during which the student is off task but is sanctioned to do so by the training 
sponsor. Nonsanctioned, off-task behavior includes such activities as socializing that is not joh 
related and is not sanctioned by the training sponsor. 

The observation form can be designed to gather information on behaviors of special interest to 
you. or you can reproduce a version of the form given here. Following are definition!} of each on- 
task and off-task category included in the form. 
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Figure 5. Time use assessment observation form for manual calculation 
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Time on task* 



o Technical Skllit«-Practlce Only refers to tasks related to the technical requirements 
of the student's position, as defined in the training phn. that do not require use of a 
basic skill (reading, computing, writing, or communicating). 

p Technical Skills— Practice Using Basic Skills are tasks related to the technical 

requirements of the student's position, as defined in the training plan, that do require 
use of a basic ekill. 

o Technical Skills/Theory— is the category of activities including receipt of instruction 
from or discussion with any employee at the training station regarding theory 
involver" in performing a task. The task is related to the technical requirements of the 
student's position as defined in the training plan. 

o Basic Skills*— Separate refers to instruction in basic skills as required by the training 
sponsor during the student's work site hours. The student is not performing any 
tasks related to technical skills practice. 

o Employablllty Skills may fall into threelcategorie^: 

—Work attitudes are taught when the training sponsor is communicating to the stu- 
dent regarding those qualifies deemed worthy by employers and fellow workers 
in performance of a job. Examples are getting to work on time, doing one's work 
well, and relating to co-workers and management in a cordial, respectful manner. 

—Knowledge of the world of work is addressed when communication from the 
training sponsor to the student contributes to understanding of how jobs are 
structured and how people prepare to engage in work. Examples include discus- 
sion of current and future job opportunities and of the social and personal Impli- 
cations and requirements of specific jobs. 

— Job-seeking, maintaining, and advancing skills are taught through communica- 
tion from the training sponsor to the stOdent regarding job placement, both on an 
initial and an advanced basis. Examples include learning new job skills to get a 
different job or position and learning how to develop a resume. 

o Entrepreneurs^ «p Skills encompass communication from the training sponsor to the 
student regarding the organizing and operating of a business venture. 

o Setting Up involves activities performed by students to prepare for performing a task 
related to the defined technical requirements of the position. 

o Cleaning Up includescleaning the work station and putting tools and materials back 
in their proper place after the performance of a task defined in the training plan. 



Time off task— sanctioned. 

o Setting Up encompasses activities the student performs in preparation for an 
assigned task nor related to the tasks defined ir« the student's training plan. 




o Cleaning Up involves assigned tasks not defined in the training plan, that are per- 
formed by the student and involve cleaning the work station or another work site 
area. For example^ the machine shop student is asked to sweep out the boss's office 
or wash windows. 

0 Rettroom Break is a scheduled or unscheduled absence from the training station to 
visit jlhe restroorn. 

o Scheduled Breaks are lunch or coffee breaks that are part of the regular work day 
schedule at the work site. 

o Waiting for Directions is time spent waiting for the training sponsor to provide Infor- 
mation or answer questions relating to a specific work task. 



e Time off task*— nonsanctloned. 

o Doing Nothing is Inactivity on the part of the student that is not sanctioned by the 
employer; there are on-task activities the student could be performing at this time. 

o Non-Job*related Socializing includes socializing with other employees or non- 
employees (not including customers) when such behavior is prohibited by the 
employer. Requirements of the employer and often the training plan Indicate that 
there are on-task activities the student could be performing at this time. 

o Disruptive Behavior is behavior that is clearly disrupting the work being conducted at 
the work site and would obviously not be sanctioned by the employer. Examples are 
fighting and horseplay. 

Note also that the form includes space at the bottom for writing notes or comments describing 
in more detail the specific activities the student is perfoiming, the work environment, and the 
apparent quality of the student's experience. These notes are discussed more fully in "Adding 
Comments" later in this chapter. 



Adapting the Form 

The observation form need not be used exactly as It is presented in this handbook. It can be 
adapted in many ways, depending upon your needs. For instance, an observation form can have 
only three columns: one each for on-task activity; sanctioned, off-task activity; and nonsanctioned. 
off-task activity. Such a form would not specify student activities but would provide more space for 
making notes. 

The form can also be adapted to be more specific. It can be tailored to the student's service 
area cr to the student's occupational role. It can reflect the individual student's training plan, 
including each of the tasks outlined in the plan. Experience In observing may suggest other adap- 
tations that would be helpful In gathering information to meet your objectives. 
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Who Should Observe? 



Information on a cooperative vocational education student's time use at the training station 
can be gathered by any of several people. One possibility is to observe the student yourself. Since, 
as coordinator, you are not a direct participant in the training process at the work site, you are less 
likely than the student or training sponsor to be biased in recording Information. Firsthand obser- 
vation also allows you to see the student's performance. Instead of relying on Information provided 
by sonneone else. The main disadvantage of collecting time-use data personally is the demand this 
places on your time. It Is also possible that your positive or negative feelings about a student or 
training station might color your perception; if you think this might happen in d particular instance, 
it is best to get someone else to conduct that observation. 

The student's cooperative vocational education teacher is also a good candidate to conduct 
the observation, since these staff members have the background necessary to understand the 
training station tasks. Like the cooperative education coordinator, the teacher should be alert for 
any personal feelings about a student that might bias the observation. In some instances, another 
vocational education teacher may be the most unbiased person available to conduct the observa- 
tion. 

Students can collect time-use information, themselves by keeping a record of their own activi- 
ties. This may be the least obtrusive way to obtain time-use information and offers the additional ' 
advantage of helping students understand how they spend their time on the job. The obvious dis- 
advantage of using students as self-observers is that they are not unbiased but are direct partici- 
pants in the process they are recording. 

Finally, information on students* time use can be collected by their training sponsors, although 
some training sponsors feel they are too busy to act as observers. Because they are direct partici- 
pants in the training process, training sponsors may be biased. Balancing this possibility, however, 
is the fact that the training sponsors thoroughly understand the tasks and processes at their work 
site, and can therefore record them accurately. Volunteer retirees are another possibility. There are 
many qualified individuals who seek active roles in society after retirement. They can serve most 
effectively as time-use observers. An excellent resource describing procedures for recruiting 
volunteer retirees is the following: 

Warmbrod. C. P.. and Eisner H. R, Operating a Retirees' Volunteer Program in Postsecondary 
Institutions: A Resource Handbook. Columbus: The National Center for Research in 
Vocational Education. The Ohio State University. 1979. 



Setting Up the Observation 

When an outside observer plans to collect time-use data, it is important to set up the observa- 
tion with the training sponsor in advance. Because you may otherwise encounter resistance from 
the training sponsor, it is necessary to explain why you want to collect time-use information on the 
student and to discuss with the training sponsor any means by yvhich the observer can remain 
unobtrusive. 

In terms of collecting accurate, unbiased data, it Is usually helpful to gain permission to 
observe at an unspecified time. However, the training sponsor may dislike this Idea. In that case, 
set a specified time for the observation to take place. 
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Observer Training 



It is important that you and any other observers you enlist be consistent in recording the activ- 
ities observed. This requires that observers have the same understanding of how each activity is to 
be recorded on the observation form and that they record similar activities consistently over time. 
It is extremely important in assessing the reliability of observers in the coding of the specific 



There are a number of ways to calculate observer agreement, Johnson and Bolstad (1973) 
state that an overall agreement of 80-85 percent is a realistic upper limit, Foi a detailed discussion 
on the assessment of reliability, the reader is referred to the following resources: 

Hawkins. R., and Dotson. V, ''Reliability Scores that Delude: An Alice in Wonderland Trip 
through the Misleading Characteristics of Interobserver Agreement Scores in Interval 
Recording." In Behavior Analysis: Areas of Research and Application, edited by U, Ramp 
and G. Semb. New York: Prentice-Hall. 1975. 

Medley. D. M, "Systematic Observation." In Encyclopedia of Educational Research. 5th ed.. 
vol, 4, edited by H. E. MItzel. New York: The Free Press. 1982. 

Siedentop. D.: Tousignart. M.; and Parker. M. Acadernic Learning fime^-Physical Education: 
Coding Manual, rev. ed. Columbus: School of Health. Physical Education and Recreation, 
The Ohio State University. 1982. 

We recommend that observers be trained through both discussion and practice observations. 

The National Center has Issued a workshop guide for training observers to assess time use. 
Because observer training does involve understanding the concepts and method of time use anal- 
ysis, we recommend this workshop guide. Managing Learning Time: A Professional Development 
Guide (Halasz and Raftery 1985). which Includes overhead transparencies and handouts, and 
which was designed as a companion to this handbook. 

Observers need to understand the purposes for collecting time-use data and how the data will 
be used. They also need to understand the specific definition of each on- and off-task activity on 
the observation form. The best way to meet these needs Is through discussions and practice 
observations, which help develop skill and consistency in observation. Practice observations can 
be conducted at training stations or in a classroom in your school with the observers recording the 
activities of one student. The observers should all record at the same time, but each should record 
the time-use information independently. After the observation, they should meet to discuss their 
data and calculate results. 



Unobtrutivenest in Observing 

In conducting an observation of a student at the training station, it is important to be as unob- 
trusive as possible; the more noticeable the observer, the more likely it is that the student and the 
training station in general will not function normally. Observers should dress in a way that will 
blend in at the workplace and should sit or stand out of the way. They should not interact with the 



behavior. 



Observing 
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student being observed, unless the student is on a scheduled break, and they should not talk with 
other workers at the training station. If spoken to, they should reply briefly, keeping interaction to 
a minimum so as not to keep the workers from doing their jobs. In general, observers should be 
careful not to get in the way of any work being conducted at the training station and should not to 
call attention to themselves. Webb et al. (1966) provide an excellent source of detailed information 
on unobtrusive measures. 



Materials Necessary for Observing 

/ The observation form shown in figure 5 will hold 15 minutes of minute-by-minute time-use 

observation. An observer recording every minute will need four forms for every hour of observa- 
tion. A clipboard should be used to hold the observation forms. Since it is awkward to look at your 
watch and often inconvenient to look at a wall clock, an inexpensive stick-on or cllp-on quartz 
clock should be used to keep track of the minutes. Finally, the observer should have several sharp- 
ened pencils with erasers for any necessary changes. 

Time Intervals 

Time-use data can be recorded every minute, every third minute, or every fifth minute. Studies 
conducted by the National Center have found no difference In the accuracy of time use data 
recorded at those three intervals. Recording time use information at 5-minute intervals allows more 
time for making notes than does recording Information every minute. 

Another option is to record time-use data every minute for 15 minutes and then to make notes 
for 15 minutes. This procedure can be repeated for the course of the observation. Regardless of 
the time interval used in recording the information, it Is Important to remember that only one on- 
or off-task category should be checked per interval. Also, whichever option is chosen should be 
used for all observations. 



Adding Comments 

The suggested observation form provides room for notes. In general, these notes or comments 
should deal with anything significant that cannot be shown in the designated time-on-task cate- 
gories Comments can be used to identify both positive and negative attributes of the work envi- 
ronment that seem to influence the time use of the cooperative education student. They might deal 
with safety factors, placement of equipment, difficult personalities at the training station, apparent 
racial or sexual discrimination, work overload, and so on. They can also be used to explain un- 
usual Interruptions or events that cut down the students' Wme on task, such as equipment break- 
down or problems with materials. 

Remember that positive factors are also important. Comments should particularly include any- 
thing that contributes to a good learning experience: advanced, well-designed equipment: a variety 
of tasks: friendly, helpful co-workers; the opportunity to solve problems, and so on. Of course, 
interactions between the student and the training supervisor are important. Note the facts, not 
your interpretation of them. Rather than "Supervisor dislikes student," note "Student requests 
help— supervisor yells at that person for stupidity." 

The sample form suggests that observers note critical incidents, critical comments, and critical 
work site characteristics. In this context, the word "critical" does not mean "negative." Rather, it 
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refers to those events that may be significant in shaping the student's learning experience. The fol- 
lowing definitions are presented to provide further definition and guidance in preparing comments 
on the observation form: 

• Critical incidents are those that affect the student's work or demonstrate something espe- 
cially positive or negative about the training situation. If, for example, a supervisor takes a 
good deal of time to explain how to do something or, on the other hand, does not respond 
to the student's request for help, this information will be valuable in making decisions 
about the training station in the future. 

• Critical comments are appropriate when observers have perceptions that seem important, 
even though they are not strictly objective. You may draw a conclusion about the overall 
tone of a workplace— it seems friendly or hectic— or you may note that the student seems 
to be excluded by co*workers. Be careful in making these remarks since they are judg- 
ments. Whenever possible, try to validate them with data. For example, the comment 
"Training Sponsor encourages independence" can br 9ccompanied by a note on a signifi- 
cant incident: "Sponsor asks student to draft a letter to patients on billing procedures." 

If interactions, events, or expectations at the work site are unusual, note them. A health 
intern may be expected to assume unusual responsibility in responding very quickly and 
competently to an emergency code. A student working in small business may be expected 
to participate in decision makirig. These demands and opportunities significantly affect 
the learning experience and should be noted. 

• Critical work site characteristics are aspects of the physical work site that interfere with or 
enhance the learning experience and should be noted. Problems might include extremes 
of temperature, faulty equipment, or unsafe practices. Poor ventilation and excessive 
smoking or dust may be annoying: if so, a note should be made. By the same token, an 
unusually well-furnished office may provide an ideal environment for working and learn- 
ing. These significant characteristics of the work site should be noted. 

Comments are a valuable part of the observation and can almost always provide a richer 
description of the demands of a cooperative education training station than you had before analy- 
sis. Some observers take copious notes: others take few. Some make only objective comments. 
Others respond to human interactions or the physical environment with unusual sensitivity: still 
others are highly aware of the work processes at a training station. Remember that the notes you 
take will be filtered through your own perceptions and your beliefs as to what is most important. 
Keep in mind however, that the focus of all notes should be the kind and quality of training the 
student is receiving. 

It is most important in making comments about training to be objective. Tie the note in with 
something actually seen in the workplace— the event that caused you to make a note. Facts about 
the training situation are more useful than inferences and judgments. These will follow later, when 
you deal with the results. 

We have found it useful to type the handwritten notes from an observation session: you might 
want to keep a copy in the student's file and a copy to be kept with the file on the training station. 
Following are examples of notes taken at two observations by different researchers. They show dif- 
. ^rent emphases, but both sets of notes deal with tangible aspects of the training station. As is evi- 
dent in reading them, they contain information that will be very useful in dealing with problems, in 
placing students at those training stations in the future, and in evaluating the quality of the stu- 
dent's learning experience. 
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Site 1: Welding Shop— Observer*! Comments 



Site 1 is a welding shop that offers supervised training to a trade and industry student. The 
shop is located in a large, unheated garage. This observation took place on a very cold March day. 

The notes show high consciousness of safety, the work environment, the specific; tasks the 
student gets to do. and the specific tools the student uses. The latter two categories of information 
are very useful in placing trade and industry students in relevant training stations. 

The notes represent 1 hour of observation. They are given as they appeared on the observa- 
tion form, with S standing for the student and TS for the training supervisor. (When more than one 
student works at a site, numbers can be used— Si, S2, and so on.) Each block of notes was taken 



8:15-8:29 S fabricating an oil recovery tank— grinding welds; operates electric arc welding 



machine. 

Wears safety glasses and gloves. 

Lighting dim but okay for welding. 

Air exhaust system running— noisy. 

TS gives instructions on welding the reservoir section. 

S finishes welds, grinds slag. 



8:30-8:59 S grinding slag. Sparks from grinding fly into welding equipment and paper 



boxes. 

Sparks fly onto TS. S moves so sparks are shielded. 

S moves "job" into position to do heavy grinding. Has to strain— job looks quite 

heavy. Scrapes welds with hand scraper. No safety glasses. 
S grinding slag. 

TS tells S to be careful of grinding sparks. S puts on glasses. 



9:00-9:14 Customer comes in, exchanges greetings with S. S opens garage door. Cus- 



tomer backs truck in. S helps load a coal stoker made in the shop into 
truck. 

S does layout work— measures for parts, using slide tape and scriber. 

Asks TS a question. TS stops work to explain. 

Door still open. Thirty-two degrees out— I can see my breath in here. 



These notes give a good deal of useful information about this training station. The training 
sponsor is available and on-site. He reminds the student of safety practices. He also permits the 
student to work alone until help is needed. The student at this training station gets practice in 
standard welding skills and must do layout work. He must also be able to do heavy lifting when 
necessary, as is often the case in welding shops. On the less positive side, this cooperative educa- 
tion student has minimal opportunity for interaction with customers or co-workers. The environ- 
ment is very cold and unduly noisy, conditions that any student placed here should know about in 
advance. 



Site 2: Warehouse—Observers Comments 

The following notes were taken by another observer at a training station located in the ware- 
house ot a truck parts store. The marketing and distributive education student working here is 



from one observation form. 
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classified as a stock person, responsible for receiving and storing goods and finding them quickly 
when they are needed. The observer's notes show, however, that several "hidden skills" and per- 
sonal characteristics are also demanded, including accuracy, physical strength, and the ability to 
work without supervision. 

This observer was especially conscious of specific tasks and of the human interactions of the 
workplace. These detailed notes add to the co-op coordinator's understanding of this workstation 
and are balanced by the objectivity of the observation form. In turn, the notes supply information 
that cannot be given in the precise categories of the form. Although both observers' notes are fac- 
tual, the difference in emphasis is apparent. 

1:00-1:14 S checking parts on microfiche reader. Works without supervision. 
TS is always in sales room, working with customers. 
S busy and involved. Seems to know exactly what to do and how to do it. 
Warehouse large, drafty, everything dusty or oily. No place to sit at all. 
Completely male work site. 

S finishes inventory, moves boxes, begins to unload. Some parts are very heavy. 

1:1 5-1 .29 S helps another employee nail together a large frame. They chat as they work. 

S seems to have excellent relationship with other employees, who say hello or 
stop to chat. 

S appears to have as much responsibility as other employees— does the same 
work. 

Continues unpacking, marking items on packing slip. 

NOTE: Owner stops to chat with me. Says S is "great" but not all co-op students 
work out as well Says co-op supervisor is good— calls weekly, handles 
problems immediately. Owner plans to hire S full-time when S 
graduates. 

1:30-1:44 S says he doesn't help customers at the front counter because he doesn't know 

how to work computer yet. 
Men here do a lot of recognizing each other in passing— "Hey. Rudi" and 

"What's new, Jim?" Friendly but very work-oriented atmosphere. 
S checking in stock. 

NOTE: Much of S's responsibility is keeping track of things. Also has to read 
and interpret a great variety of packing slips, shipping labels, order 
forms, etc. 

Stops to help another employee load an order onto a truck. Much heavy lifting. 



1 45-1 :59 This is pretty boring work, as far as I can see. but S doesn't seem to mind. He's 

very energetic and works hard. 
S helps another employee nail up boards on a shelf unit. They talk while 

working. 
S asks TS how to do something. 
TS traps S in a box. teasing. 

Then back to work, rolling rubber tubing onto cylinders. (They tease and have 
fun as they work.) 

NOTE: Most of S's time is spent logging parts in or out. These records are used 

to reorder when something runs low. so accuracy is very important. 
Co-op supervisor drops by to say "hi" to S and his boss. 
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These notes demonstrate the "feel" for a training station that can only be gained through an 
observation. They also show the wealth of information that can be gathered in a relatively short 
time. The focus of a time-use analysis of a training station is usually the recording of the student's 
activities on the observation form. Most people find, however, that collecting specific time-use data 
rapidly becomes easy and that they are able to add significant comments that broaden their 
understanding of the student's learning experience. 



Calculating the Results 

Once the time-use information has been collected, you will want to calculate the results for the 
purpose of interpretation. Calculating the results is a two-stage process that consists of first com- 
puting the information and then displaying the results. 



Computing Time Use Information 

The process is essentially simple. An analysis of the student's time use at the training station 
can be calculated using figure 6. If, for example, you observed a student every minute for a 1-week 
period, the total number of minutes of observation would be 1,005 minutes. If the student spent 
391 minutes on technical skills practice, then the percentage of observed time th it the student 
spent in technical skills practice would be 391 -r 1,005 = 38.9 percent. A percentage should be cal- 
culated using this method for each time-on task and off-task category. Worksheet 1 found in the 
Appendix can be duplicated and used as a guide to compute time use. 



Summarizing the Results 

Once time-use findings are calculated, they can be examined through use of figure 7. This 
worksheet directs attention to important evaluation questions, Including questions that may be 
dealt with in observer's notes. Worksheets found In the appendix can be duplicated and used as a 
part of your summarizing the results. 

This worksheet is useful in talking over results with those affected by a specific time-use 
assessment and can be used in conjunction with a graphic presentation, such as the bar chart 
described in the next section. 



Displaying the Time-Use Results 

The primary purpose for displaying time-use results is to present information in a concise, 
comprehensible manner. Graphic presentation is an extremely useful and efficient medium for the 
presentation of time-use data, and is often more appropriate than narrative presentation. 

• It is more effective for creating interest and catching the attention of the reader. 

• It illustrates visual relationships, which are more clearly grasped and more easily 
remembered. 

• It i' more efficient, since the essential meaning of masses of statistical data can be assimi- 
lated at a glance. 
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Total minutes observed 



Identification Codes 



Student: Jchn Smj-fh 



Training Sponsor; Ttle. dammnni Ka-HonA 

Observer: Oiir/oA eSfc/aa.^ 

Observation Date: /z/j^/ftj 



Tatk 

Category 



Total 
Minutes Observed 



Percentage 
Of Observed Time* 



1. Technical skills practice 

2. Technical skills w/basic skills 

3. Technical skills/theory 

4. Basic skills 

5. Employability skills 

6. Entrepreneurship skills 

7. Setting up (on task) 

8. Cleaning up (on task) 

TOTAL TIME ON TASK 

(add percents 1 through 8) 



31L 
ZhS 

L 

ML 



-JL 
ML 

101 



IMS 
/ocS 

ms 

/oo£ 



9. Setting up (off task) 
10 Cleaning up (off task) 

11. Restroom break 

12. Scheduled breaks 

13. Waiting for directions, etc. 



-a 

201 



IDOS 
l£OS 



a2% 



TOTAL SANCTIONED TIME OFF TASK 

(add percents 9 through 13) 



I606 = 2S.1 O/n 



14 Doing nothing 

15. Non-job-related socializing 

16. Disruptive behavior 

TOTAL NONSANCTIONED TIME OFF TASK 

(add percents 14 through 16) 



/ooS 



10^ = UL 



% 



Figure 6. Sample form for computing time use 



27 



ERIC 



J7 



Student: J^h^ Stvit-^A Observation Dale: ^^/^^/ f f 

Training .^tatinn /^j^pfit^uni /(^A-^^J/ins^ Observation Tirre: /'fiC/im. -^'OOp.^' 

DIRECTIONS: Duplicate this worksheet, giving copies to the cooperative education coordinator, 
teacher, observer and others involved in interpreting the results. Refer to the connpleted observa- 
tion forms and worksheet 1 to answer the following questions. 

1. What percentage of time did the cooperative education student spend on task? 7^- H percent 

2. What percentage of the observation time did the student spend on specific on-task activities? 
38'^^ Tftrhnirai skills practice only 

Technical skills practice using basic skills 
.^5ii^Technical skills/theory 
Q Basic skills 

0. /^ Fmplnyahility skills 
-A^^Entrepreneurship skills 

V^ .^tting up 
/^/T rjpaning up 

3. What percentage of the time did the student spend on specific sanctioned, off-task activities? 

J.^% c;ptting up 
A^CIeanlng Up 
«Zj^^Restroom break 
.l^i^Scheduled breaks 
^^iZ^Waiting for directions 



Figure 7. Sample form for tummarizing time-ute findlngft 
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4. What percentage of the time did the student spend on nonsanctioned. off-task activities? 

Doing nothing 
.^J^Non-job-related socializing 
Disruptive behavior 

5. Did the student^s training station activities rellect the learning activities designated in the 
training Dian? 

Jes^Yes No 

! 

6. How much responsibility did the training sponsor have in the cooperative education student's 
training station activities? 

-jUt^ Very much Some Little 

7. Did the student display safe work practices? 

Yes JI^\\o 

8. Did the cooperative education student work independently while at the training station? 

^J^^^TYes No 

9. What type of working relationship existed between the cooperative education student and the 
regular employees?. 

/ 

Independent Supportive 

10. Did the training station activities provide the student with the opportunity to enhance his pro- 
ficiencyjn occupational or job-specific skills? 
r Yes No (If no. please explain) 

Cooperative Education CQordlnator ^S/^sa^ Lee. 

Observer: 
Observer Title: 



Figure 7— Continued 
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• It provides a more comprehensive picture of a problem than figures alone, enabling a 
viewer to gain a more complete and better balanced understanding. 

• It may stimulate and facilitate analytical thinking and investigation imto hidden facts and 
relationships. 

Program evaluation Information in particular can be more effectively presented using graphic 
displays. Detailed descriptions of ways to display information can le [ -^ur.d in the followincj 
publications: ^ 

Franchak. S. J., and Kean. M. H. Using Evaluation Results, Columbus: The National Center for 
Research in Vocational Education, The Ohio State University, 1981. 

Holley. F. M. A Communication Handbook for Researchers and Evaluatcrs. Austin. TX: Austin 
Independent School District, 1979. 

Schmid. C. F.. and Schmid. S. E. Handbook of Graphic Presentation. New York John Wiley 
and Sons, 1974. 

Starr. H.; Maurice. C; Black, M.; and Keller. P. Selecting. Analyzing, and Displaying Planning 
Information. Columbus: The National Center for Research in Vocational Education, The 
Ohio State University, 1979. 

One of the most commonly used graphic presentations is the bar chart. There are at least 
eight types of bar charts; for displaying time-use data, the simple bar chart can be quite effective. 
Figures 8 and 9 are provided for the purpose of developing a graphic display. The specific time-on- 
task and off-task categories have been inserted line by line down the left side of the chart. The 
percentage of time the cooperative vocational education student spent performing tasks in each 
category can be displayed by darkening the lighter dotted lines to correspond with that percent- 
age, as measured on the horizontal axis. 

Although time-use assessment is unfamiliar to many people, the use of the summary sheet 
(worksheet 2) and graphics displays should enable students, training sponsors, and others to 
grasp the meaning of the results quickly. 



Summary 

Conducting a time-use study involves three basic steps: preparing, observing, and calculating 
results. Some key concepts to bear in mind as you prepare for your first time-use analysis are 
given in figure 10. 



I 
I 
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Student: dohfi Sml-f/l 

Training Sponsor: 7g/» ^yammuni /\a4nion^ 



Observer: 



Observation Date: fX^^s/j V 



DIRECTIONS: Use copies of this worksheet to display graphically the student's time use summa- 
rized on worksheet 2. The specific time-on-task and off-task categories have been listed on the left 
side of the sheet. Develop a bar graph by d arkening the dotted line for each category to corre- 
spond with the percentage (measured on the horizontal axis). 



Task 

Categories 

Technical Skills 
> Practice Only 

Using Basic Skills 
Tfechnical Skills/Theory 
Basic Skills 
Employability Skills 
Entrepreneurship Skills 
Setting Up 
Cleaning Up 
Total Time on Task 




0 10 20 30 40 50 60 70 80 90 1 00 
(Percentage of Time on Task) 



Figure 8. Sample form for developing a time-on*task bar graph 
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Student: Jnlin cJ^yy/// 



Training Sponsor: ~7aU ^fimnounj / (g-h/^n.'^ 



Observer /Jar/A!^ K^/^aS 



Observatio n Date: /^//S/ ^^ 



'""^'..S , 

DIRECTIONS: Use copies of this worksheet to display graphically the student's time use summa- 
rized on worksheet 2. The specific time-on-task and off-task categories have been listed on the left 
side of the sheet. Develop a bar graph by darkening the dotted line for each category to corre- 
spond with the percentage (measured on the horizontal axis). 



Task 

Categories 



Sanctioned 

• Setting Up 

• Cleaning Up 

• Resfroom Break 

• Scheduled Break 

• Waiting for Directions 

Nonsanctioned 

• Doing Nothing 

• Non-Job-related 
Socializing 

• Disruptive Behavior 
Total Time off Task 




10 20 30 40 50 60 70 80 90 100 



(Percentage of Time off Task) 



Figure 9. Sample form for developing a time-off-task bar graph 



1 
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• Your objective will indicate which 
training stations should be observed. 

• The observation form can be designed 
to gather information on behaviors of 
special interest to you. 

• Information can be gathered by any of 
several pec<p!e. 

• Observers should be careful not to oet 
in the way of any work being con- 
ducted at the training station and not 
to call attention to themselves. 

• The focus of all notes should be on the 
kind and quality of training the student 
is receiving. 

• The primary purpose for displaying 
time-use results is to present informa- 
tion in a concise, comprehensible 
manner. 

V — 

Figure 10. Conducting a time-use study—key concepts 
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CHAPTER 4 



WORKING WITH THE RESULTS 



Gathering Additional Information 

If your observation of a student at a training station is limited to one or two short visits, you do 
not have a representative sample of that student's training station experience. It is therefore impor- 
tant to gather additional information. Three basic sources of more Information are available: the 
student, the training sponsor, and the student's vocational education teacher. In all likelihood, the 
information from these sources will be colored by opinion and judgment and should be evaluated 
carefully. Nevertheless, these discussions can add valuable feedback, particularly on subjective 
topics, such as the quality of the student's performance at the work site. 

These interviews for additional information can be structured or short and casual, depending 
on the time you have available to conduct them. Specific questions ("Do you usually have as much 
trouble operating the copy machine as you did today?") generally elicit more information than 
general questions ("How's it going?"), Answers should be written down and kept on file. 

It is often easy to talk to the student in person at school, perhaps before or after the related 
class. Other students who have worked at the site may contribute valuable Insights as well. 

Time constraints may dictate that you talk with the training sponsor by telephone. Avoid 
implying that you are evaluating the training station by emphasizing your interest in the student's 
learning experience. In talking with the work site supervisor* ask questions that specifically deal 
with the student's learning and relate to the tasks specified In the training plan. If. for example, the 
student works in an office, some relevant questions might ba— 

• "How much time does the student spend at the word processor?" 

• "Is the student dependable?" 

• "Has the student taken on any new tasks recently?" 

• "What has the student learned since the last time we talked?" 

The cooperative vocational education teacher will share information from a third perspective, 
which will vary with the structure of your cooperative education program. If the co-op student con- 
tinues to receive formal training, the teacher will have information on progress In skills. If the stu- 
dent's formal education is now confined to attending the related instruction class, the teacher will 
probably have insights derived from the student's comments about the training station. Class dis- 
cussions may Indicate that the sludent Is bored, challenged, or over supervised, Is receiving too 
little instruction, feels overworked, or is underutilized. 
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Talking to the student, the training sponsor, and the co-op teacher is a good way to gather 
more information about the student's learning time at the training station. Open communication 
serves many purposes. In terms of the objectives of this handbook, communication is important 
because it keeps Information flowing to the co-op coordinator. This, in turn, makes possible more 
effective evaluation, better program management, and program improvement. 



Evaluating the Learning Experience 

Cooperative vocational education exists to provide what the formal classroom cannot. This 
may include experience on various kinds of equipment and machinery not available in the class- 
room, such as large switchboards. It may include the opportunity to practice service skills, such as 
waiting tables or caring for children, that can be taught in the classroom but that cannot be expe- 
rienced outside the actual setting. Whatever the nature of the training station, it should offer a 
learning experience that is an extension of classroom learning. Evaluation should, therefore, begin 
by asking whether the student's tasks at the training station are clearly related to the student's ser- 
vice area. Is the student learning skills that consolidate, reinforce, or extend classroom learning? 

lA second area for consideration is how well the student's time use corresponds to that shown 
in the training plan. Ideally, the student is performing the specific tasks detailed in the plan, but 
this is not always the case. Even when the training plan is being followed, observation may suggest 
that the student is no longer learning from the work, and that the training plan should be modified. 
(How to modify a plan is discussed later in this chapter.) 

The student's time on and off task should be examined in view of conditions at the specific 
training station. It is important to note the difference between sanctioned and nonsanctioned time 
off task, since only the former is created by the demands of the job (such as setting up and clean- 
ing up) and permitted by the employer. The student's time on task should be compared to that of ^ 
other students in the program and in the service area or occupational category. Our study of time 
use at the training station (Franchak and Norton 1985) examined the time use of 40 students. 
Because of the study design, the results should not be generalized, but that may be of interest. 
Overall, the students were on task about 90 percent of the time. Only about 1 percent of their off- 
task time was spent in nonsanctioned. off-task activities, and the other off-task time was sanqti- 
oned by the work supervisor. These off-task activities included scheduled breaks, waiting for direc- 
tions and so forth. 

The remainder of your evaluation will rest on your criteria for co-op student learning. These 
standards must be defined by the individual co-op coordinator, with input from other educators 
and local or state agencies as appropriate. 

In our study, a content analysis of the notes observers took during their observations of train- 
ing stations suggested that six factors were related to the quality of the training station experience. 
These factors are given below, for possible use in defining evaluation criteria. 

• Type and variety of tasks performed. Do the student's tasks offer the opportunity for both 
challenge and mastery? At high quality training stations, students seemed to be assigned 
tasks that enabled them to learn a variety of skills and remain Interested in their work. 

• Level of supervision. Students show varying needs for supervision, and training stations 
usually have distinct orientations toward more or less supervision. Whether the level of 
supervision is adequate should be evaluated in context of how much supervisioii the indi- 
vidual student needs to learn well. 
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• Extent of instruction. Instruction should be sufficient to guide the student through diffi- 
cult and frustrating tasks, but not ever-present so that the student huS no opportunity to 
demonstrate initiative. 

« Amount of work. Students can have time off task because they are not given enough to 
do. The amount of work given a student should be neither so little that the student stands 
idle nor so much that the student is rushed and frustrated. 

• Conditions at the work site. Inadequate ventilation, poor safety practices, uncomfortable 
temperatures, and the like may be problems in themselves that present danger to students 
and necessitate their removal. Borderline environmental problems may affect learning and 
should be evaluated for their impact and the possibility of change. 

• Supervisor's commitment to the student. This factor can give insight into the student's 
learning. When the training sponsor is genuinely committed to teaching, the student's 
learning is facilitated. Observation at the training station should determine whether the 
training sponsor or an employee designated by the training sponsor is available to the 
student for discussion and instruction. 

Your criteria for assessing student learning may be based on some or all of these factors, as 
well as on others you Identify as important. The list could include, for example, gaining familiarity 
with new tools or processes, working with others, or solving problems. Some criteria may grow 
directly out of program objectives for students in that occupational specialty. 



The Action Plan 

Evaluation is sometimes seen as a process of simply pa^ssing judgment. Instead, it is ideally a 
process heading to improvement. In observing a training station, you are likely to identify problems 
or see ways students could learn more through better time use. These problem areas can be trans- 
lated into specific goaks, as shown in figure 1 1 , which is an example of a cooperative education 
coordinator's action plan. The worksheet asks you to rank the desired changes in order of priority. 
For instance, the top priority for a business and office student working as a receptionist may be to 
increase time on task. The goal, decided in conference with the student and training sponsor, 
might be to increase that time on task from 70 percent to 85 percent. The specific strategy could 
be to give the student filing duties to carry out when the switchboard is not busy. 

The next section of this chapter discusses ways to identify problem areas that may be respon- 
sible for a student's poor time use or low-quality learning experience. These ideas may also help 
you in formulating action plans. 

Identifying Problem Areas 

Poor time use can result from any number of causes, including the student's own attitude or a 
student and training station combination that just doesn't work. However, when a cooperative edu- 
cation coordinator is conscientiously screening students and making training station decisions 
carefully, it is unlikely that problems can always be attributed to the student. They may equally 
well originate at the workplace. ^ 

Problems within a training station may be signalled by low time on task. But even when the 
student's time on task is high, there may be problems that lower the quality of the learning expe- 
rience. The conditions discussed here are not uncommon in workplaces; they have already been 
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DIRECTIONS: Use this worksheet to decide what changes you will make to enhance the coopera- 
tive education student's learning experience. Refer to worksheet 2 for information about the cur- 
rent use of time at the student's training station. In developing goals and strategies, consider 
school and work site policies and any other potential limitations. 
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Observed use of 
co-op student time 


Goal for student 
use of time 


Specific strategies 
for reaching goal 


1 


dees no^/un^ /cn^ 




clij4fe6 iv carry oui' kjHin 
^aji-fchbMrd i^fic-f J^sy. 


2 


S-fodtn-f- Socialists 
U^ina phont A>r 
nonousintss p4/rfi06cs 


^ crease 47ntt cn -hts^ 


afirHes^ carry ^ lOftin 
3i/jMibHrDl is no^ busy 


3 








4 








5 









Cooperative Education Coordinator . 

Student ^J/Q^r7 Smi-f-h 

Date 

Training Station . 



Figure 11. Cooperative education coordinator's action plan 
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observed in studies of time use in the vocational-technical classroom (Halasz and Behm 1983: 
Halasz. Behm. and Fisch 1984). 



In evaluating a student, it may be helpful to consider whether any of the factors discussed 
here are present, either at the training station or in the student's practice or attitude. If they are. it 
is possible to work toward specific changes that car increase the student's time on task and 
improve the quality of the learning experience. 



Poor Organization and Planning 

Time on task is lowered by such problems as faulty equipment and inadequate or misplaced 
tools and supplies. In some workplaces it is almost taken for granted that one machine will brbak 
down every week. Sometimes this means the student does "make-work" with the files or sits idle 
until the typewriter is repaired; sometimes it means a lost hour trying to get a poorly cleaned paint 
sprayer to work properly. Such pursuits, although defined as sanctioned, off-task activities, are not 
very productive. 

Day-to-day disorganization can be equally hard on students. Disorganization is sometimes the 
cause of excessive setup time, as the student searches for materials that seem to have dis- 
appeared. General disorganization can also lead to the student's spending a high amount of time 
waiting for directions. A training sponsor's disorganization Is sometimes signalled by the fact that 
workers and co-op students stand idle, waiting for attention to problems and for work 
assignments. 



Low Value Put on Student's Time 

Students who are often off task or spend a great deal of time "fooling around" with setup or 
cleanup may have no idea that time is important. Some students in this category tend to become 
deeply involved in whatever they are doing, for example sweeping their work areas, and doing it 
too thoroughly and for too long a time. Others do not have clear images of the work goals to be 
met and the time it is supposed to take to meet them. For these students, Parkinson's Law really 
has truth; their work expands and fills all their time until someone imposes a deadline. 

Firmer goals can be of great help in such situations, as can deadlines imposed by the training 
sponsor. The problem is sometimes appropriate to discuss with both the student and the training 
sponsor. When cooperative education students do not seem to value their work time, it may be that 
their training sponsors do not place high value on the students' time either and so have not pushed 
the students to be more productive. A key element of cooperative education is the students' pro- 
ductivity, a point that can only benefit the training sponsor who understands it. 



Undefined Goals or Expectations 

Complaints about a student's work or poor performance ratings sometimes result from the 
training sponsor's failure to communicate goals and expections clearly. For instance, a student 
might assume that in word processing a bulk mailing, absolute accuracy is required. The 
employer, on the other hand, may feel speed is the top priority, may expect a few errors, and may 
ultimately be very frustrated with the student's slow pace. 
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students can become discouraged and slack in their work when they find they have not done 
the task well enough, soon enough, often enough, and so on; that is. they have not met the employ- 
er's hidden expectations. Educational research has shown that when goals are clearly.defined in 
the vocational-technical classroom, students are on task more of the time (Halasz. Behm. and 
Ftsch 1984). The value of deadlines and written goals and objectives in the workplace is also well 
known, and much management theory deals with defining objectives. 

Hidden expectations are so common that sometimes an employer's complaint about a stu- 
dent's behavior or work surprises the student. Students may have no idea that there is al dress code 
at the work site.^ production quota to meet, or an expectation that slack time will be used to 
organize materials or clean equipment. Older, more experienced workers and supervisob often 
feel students ought to know these things, but some students don't. One of the most helpful prac- 
tices a training supervisor can have is that of clearly and directly telling students what is iexpected 
of them. The great majority will work hard to meet these expectations and to accomplishlclearly 
defined work goals. 



Unrealistic Expectations \ 

\ 

Even well-defined goals and clearly expressed expectations create learning problems when 
they are not realistic. When too much Is expected of a cooperative education student, frustration 
and discouragement wilNultimately diminish learning. When too little is expected, the job is likely 
to become boring and routine. One of the best ways to keep students on task Is to assign acltivities 
that are appropriate to their level of skill and that offer some new challenne. 

If observation suggests that a student's work lacks challenge, this may Indicate that the train- 
ing sponsor's expectations are too low. It can be helpful to outline some of the other skills the stu- 
dent has and could use on the job. perhaps to the employer's benefit. 

Students can also be discouraged by work that is too challenging. Training sponsors may 
have little background in teaching and do not necessarily know how to pace learning. Sometimes, 
for example, a student shouTd be given only part of a job in order to learn one or two of the specific 
skills involved. If the training sponsor is prone to giving overambitious assignments, it can help to 
modify the training plan to make it more specific. Detailing tasks gives the training sponsor a 
clearer map of the progress the student can logically be expected to make. In working with task 
detailing, discuss what new tasks the student should be expected to undertake at any one time. 
The detailed training plan can greatly help to shape expectations in a realistic manner. 



Unchallenging Tasks 

Sometimes, because a training sponsor does not realize the extent of a student's skills, the 
student is given a very limited set of tasks that can be quickly mastered, so that no further learning 
is involved. Even good workers tend to spend more time off task when their duties demand nothing 
of them and offer no challenge. The ideal training situation is one in which the student is given 
new tasks from time to time. Once the employer understands that the student can and should 
tackle challenging tasks, a more productive learning experience should develop. 
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Overdependence on Supervision 



Our observations suggest that most training sponsors teH cooperative education students how 
to do their jobs and trust them to proceed. Occasionally, hpwever, a student (or an entire work- 
place) is too heavily supervised and is made to be dependent on the supervisor for materials or 
step-by-step instructions. This affects morale and interfe/es with learning. 



The problem of overdependence can also originate with the student. Some students want con- 
tinual help and confirmation of the smallest decision^; not every training sponsor kno^^'s how to 
manage a situation of this kind. Gentle Intervention by the co-op coordinator can be helpful to all 
parties; both the employer and the student are losing work time. Moreover, the student should be 
learning an important skill, the ability to work independently. Overcontrol and overdependence 
work against this learning experience. 



Too Little Positive Reinforcement 

Most cooperative education coordinators have had the experience of telling students that their 
employer is very pleased with their work, only to find that the students are actually surprised. It is 
often assumed in the workplace that people are paid to do what they do and should need no 
further reward. This, however, has beepi proven to be untrue. People work harder and stay on task 
more when they get positive feedback. Young people are likely to need such feedback even more 
than adults. 

When it is difficult to pinpoint a reason for a student's low time on task, it may help to observe 
the interactions between student and training sponsor. Does the training sponsor pay any atten- 
tion at all to the student's work? Is the training sponsor's response curt and critical? If so. how 
does the student react to such'comments? Sometimes there are obvious clues that a student feels 
undervalued and demoralized due to a lack of positive reinforcement. In other cases, the student's 
response is displayed indirectly, through disruptive behavior, idle time, too much unproductive 
time cleaning up. in short, avoidance o^ tasks for which the student feels unrewarded. 



Dubious Role Models 

Often, when a student at the training station spends time in disruptive behavior, socializing, 
and just doing nothing, there are others at the work site behaivlng the same way. Supervisors and 
co-workers set the tone of the workplace, and otherwise responsible students can rapidly assume 
undesirable habits. A cooperative education student working in a small shop where service offered 
by other employees is slow may very well act accordingly, becoming slow to wait on customers. A 
student machiftist will tend to be no more precise and methodical than the immediate role models 
in the shop, and may even discard the good work habits learned in vocational-technical classes. 

Sources of low time on task are sometimes visible during observation; in other cases, only the 
results of observation suggest the existence of a problem. The difficulties described here are fre- 
quently encountered In the workplace; most are fundamental errors in management. The next sec- 
tion of this handbook discusses working to solve these and other problems identified in your 
evaluation of the training station. As In other management situations, the key is communication. 
Identifying problems in fairly specific terms should help in communicating your concerns as an 
educator to the student and the training sponsor. 
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Modifying the Training Plan 



The results of an assessment of a student's learning experience at a training station may. indi- 
cate a need to modify the student's training plan. Observation may indicate, for example, that the 
student is not performing certain tasks specified in the training plan. In that case, either the train- 
ing sponsor needs to begin assigning those tasks to the student, or the tasks in question were not 
appropriate to begin with and should be deleted from the training plan. 

it may be that the student is performing the specified tasks but spending little time on certain 
activities designated In the training plan. Time-use data are very useful in this situation, since you 
can show the employer what you observed and, using the action plan, discuss what percentage of 
time use is more desi(;able. 

We believe that specifying both the tasks to be performed and the approximate percentage of 
time to be spent on each task is desirable. If you have not done this in the past, you might consider 
beginning this practice as you modify training plans. 

Here is a list of issues that can be addressed in a training plan, gleaned from Place Co-op Stu- 
dents on the Job, a module in Category J: Coordination of Cooperative Education of the 
Performance-Based Teacher Education Module Series (Hamilton et al. 1978) developed by the 
National Center: 

• Student's career objectives and DOT code number (the Dictionary of Occupational Titles 
is a good source for determining competencies needed for occupations) 

• Competencies to be developed 

' • Broad occupational areas in which student will be trained 

• Specific tasks to be performed, in sequence 

• Time period during which each task will be performed (entire time? first week?) 

• Amount of allotted time to be spent on each task or each group of tasks 

• Related classroom study (if applicable) 

The module is a vary useful source of ideas for designing a training plan and includes sample 
plans from several States. Note that the listing merely indicates training plan concerns and is not a 
complete guide to designing training agreements. It may, however, help you assess your own train- 
ing plans and determine whether to examine other models. 



Whether the results of a time-use analysis are positive or negative, it is a good management 
technique to discuss them with both the student and the training sponsor. When results show that 
the student is on task a high percentage of the time, that instruction is available, that appropriate 
tasks are being assigned, and so on, positive feedback is in order. People who have been observed 
are often curious about the results, and good results should be shared. 



Talking Over the Results 
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When results indicate that changes should be made, a conference should be seen as a way for 
those concerned to decide together on goals and strategies for change. If you have adopted our 
previous suggestions, then comnnunication is already established with the training sponsor and 
student. Your goal for this conference should be to encourage this communication and to arrive at 
a consensus on future change. Worksheet 4 (found in the Appendix) on developing the action plan 
can be presented at this point, either blank or with the perceived problems listed. Let the employer 
and student suggest goals for change and strategies for reaching those goals. 

Follow-up is important in any attempt to promote change. Decide on a time when you will 
again stop at the training station or check with the training sponsor and student by telephone; 
probably no more than 2 weeks should elapse before this follow-up. It is a good idea to check in 
again after another similar interval. At some time within a month or two after the first observation, 
you should again observe the student at the training station. 

Checklist 1 lists strategies that are helpful in using the results of time assessment to improve a 
student's learning experience at the training station. 



Summary 

This handbook has focused on a method for assessing time use at the training stations in your 
program and using the results productively to enhance the learning experience of individual stu- 
dents and to evaluate and improve your prpgram as a whole. Figure 12 lists a few of the ideas dis- 
cussed in the final section, which emphasizes working with the results to implement positive 
change. 

The results of your analysis may also be of interest to others, including cooperative education 
coordinators and teachers, as well as State and local administrators. Time-use assessment adds 
useful information to the evaluation effort, giving specific results that can be used to work toward 
positive change. We are confident that the method presented here can help you improve your 
cooperative education program. 
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STRATEGIES FOR 



CHECKLIST 1 
USING TIMS-USE ASSESSMENT RESULTS 



Have you considered the need fo: 

1 . Identify the clients of time-use stlidy results 

2. Present results in a format ctientslcan understand 

3. Be brief in presenting results 

4. ' Emphasize the nontechnical asped^ts of findings 

5. Deal with areas of resistance 

6. Be alert to special problems inherer^t in the training station environment 

7. Encourage active client participatiorj in improving student time use 

I 

8. Arrange for follow-up assessment \ 

\ 



Yes 


No 


□ 


□ 


□ 




□ 


a 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 



SOURCE Adapted from Franchak and Kean (1981, p 64), 
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• Cooperative vocational education 
exists to provide what the formal 
classroom cannot. 

• Open communication keeps informa- 
tion flowing to the cooperative educa- 
tion coordinator. 

• Evaluation should begin by asking 
whether the student's tasks at the 
training station are clearly related to 
the student's service area. 

• PrcDiem areas can be translated into 
specific goals^ 

• One of tne best vyays to keep students 
on task is to assign activities that are 
appropriate to their level of skill and 
that offer some new challenges. 

• Follow-up is important in any attemp* 
to promote change. 
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Figure 12. Working with the results—key concepts 
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APPENDIX 



Observation Form I for Manual Data Analysis 

Obsehvation Form I A for Computer Oata Analysis 

Workshset 1: Computer Time Use 

Worksheet 2: Summary of Time-Use Findings 

Worksheet 3: Time-on-Task Bar Graph 

Worksheet 4: Time-off-Task Bar Graph 

Worksheet 5: Cooperative Vocational Educator's Action Plan 
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OBSERVATION 
FORM I 




TIME-USE STUDY FOR CO OP STUDENTS 
AT THEIR TRAINING STATION 



Identification Codes 



Datt 

Mo Dty 



li 



1 

2 
3 
4 

5 
6 
7 
8 
9 
10 
11 
12 
13 
14 
IB 





Timo on Task 


Tima Dff Task 




Technical 
Skills 
Practlct 


Technk:8l Skills/ 
Theory 


Basic Skills 


UJCO 


a 

c u 


Setting Up 


Cleaning Up 


Sanctionad 


Nonuncttontd 


Setting Up 


Cleaning Up 


E 

n 

q: 00 


Scheduled 
Breaks 


Watting for 
Directions, etc. 


Doing Nothing 


Non^lob- 
related 

Socializing 


Disruptive 
Behavior 


Tlm« 


> 
c 
O 

0 

1 


Using Basic 
Skills 
































































































































































































































































































































































































































































































































TOTALS 



































Notes: 



*OMcrib« ttudtntf' onyoint wtlvitiM it th« 
training iHt. Bt lurt to nott th« folknving: 

• Spacif ie on-t«* totivltlM 

• Critiul Incidtnta 

• Criticcl eommanti 

• CrttkMl worfc iltt diarwttrlftiet ^ 
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OBSERV/'TION 
FORM lA 




TIME-USE STUDY FOR CO OP STUDENTS 
AT THEIR TRAINING STATION 



Identification Codes 


Dttt 

Mo Day 


t 

jP 

0 


Studant 
Number 


CO 













V2 34 5 67 7^ 



1 
2 
3 
4 
5 
6 
7 
8 
9 
10 
11 
12 
13 
14 
IB 



r 


Time on Task 


Time off Task 




Technical 
8k nil 
Practica 


1 echnk»l Skills/ 
Theory 


Bask: Skills 


1 
> 


a 

1 

c ^ 


Setting Up 


Cleaning Up 


Sanctionad 


Nonsanctionad 


Setting Up 


Cleaning Up 


E 

is 

GC QQ 


Scheduled 
Breaks 


1 Waiting for 
1 Directions, etc. 


Doing Nothing 


NonOob- 

related 

Socializing 


Disruptive 
Behavior 


TInia 


Practice Only 


Using Basic 
Skills 

































































































































































































































































































































































































































































































































e-10 11-13 13 14 
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Notes: 



^Da^rlba tfttidantt' ongoing ai»ivitias at the 
training iHa. Ba sure to note the following: 

• Spaclficon-tailc activitftn 

• Critioal InqldeiiiS 
o Cr It leal eomnr>ants 

^ CrKloal work iHa eharaetarlttkM 
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WORKSHEET 1 
COMPUTE TIME USE 



Identification Codes 

Total minutes observed: 

Student: . 



Training Sponsor: 
Observer: 



Observation Date: 



Task Total Percentage 

Category Minutes Observed Of Observed Time* 



1. Technical skills^ractice -r = % 

2. Technical skills w/basic skills = % 

3. Technical skills/theory -r = % 

4. Basic skills = % 

5. Employability skills -r = % 

6. Entrepreneurship skills -r = % 

7. Setting up (on t^sk) . -f- = % 

8. Cleaning up (on task) -r = % 

TOTAL TIME ON TASK 

(add percents 1 through 8) -r = % 



9. Setting up (off task) 

10. Cleaning up (off task) 

11. Restroom break 

12. Scheduled breaks 

13. Waiting for directions, etc. 



_% 



TOTAL SANCTIONED TIME OFF TASK 

(add percents 9 through 13) 



14 Doing nothing -r = % 

15. Non-job-related socializing -f* - % 

16. Disruptive behavior - -r % 

TOTAL NONSANCTIONED TIME OFF TASK 

(add percents 14 through 16) = % 
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WORKSHEET 2 
SUMMARY OF TIME-USE FINDINGS 



Student: ^ ^ Observation Date: 

Training Station: , Observation Time: 

DIRECTIONS: Duplicate this worksheet, giving copies to the cooperative education coordinator, 
teacher, observer and others involved in interpreting the results. Refer to the completed observa- 
tion forms and worksheet 1 to answer the following questions. 

1 . What percentage of time did the cooperative education student spend on task? percent 

2. What percentage of the observation time did the student spend on specific on-task activities? 
Technical skills practice only 

Technical skills practice using basic skills 

Technical skills/theory 

Basic skilLs 

Employability skills 

Entrepreneurship skills 

Setting up 

Cleaning up 

3. What percentage of the time did the student spend on specific sanctioned, off-task activities? 
Setting up 

Cleaning Up 

Restroom break 

Scheduled breaks 

Waiting for directions 
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4. What percentage of the time did the student spend on nonsanctioned. off-task activities? 
Doing nothing 

Non-job-related socializing 

Disruptive behavior 

5. Did the student's training station activities reflect the learning activities designated in the 
training plan? 

Yes No 

6. How much responsibility did the training sponsor have in the cooperative education student's 
training station activities? 

Very much Some Little 

7. Did the student display safe work practices? 

Yes No 

8. Did the cooperative education student work independently while at the training station? 

Yes No 

9. What type of working relationship existed between the cooperative education student and the 
regular employees? 

Independent Supportive 

10. Did the training station activities provide the student with the opportunity to enhance his pro- 
ficiency in occupational or job-specific skills? 
Yes No (If no. please explain) 



Cooperative Education Coordinator: 

Observer: ^ 

Observer Title: 
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WORKSHEETS 
TIME-ON-TASK BAR GRAPH 



Student: 

Training Sponsor: 

Observer: _ _ Observation Date: 

DIRECTIONS: Use copies of this worksheet to display graphically the student's time use summa- 
rized on worksheet 2. The specific time-on-task and off-task categories have been listed on the left 
side of the sheet. Develop a bar graph by darkening the dotted line for each category to corre- 
spond with the percentage (measured on the horizontal axis). 

Task 
Categories 

Technical Skills r- 

Practice Only 

Using Basic Skills 

Technical Skills/Theory — - - 

Basic Skills 

Ennployability Skills 

Entrepreneurship Skills 

Setting Up 

CleaoingtJp 

Total Tinne on Task I , 

0 10 20 30 40 50 60 70 80 90 1 00 

(Percentage of Time on Task) 



ERIC 



54 



61 



WORKSHEET 4 
TIME-OFF-TASK BAR GRAPH 



Student: 



Training Sponsor: 



Observer: 



Observation Date: 



DIRECTIONS: Use copies of this worksheet to display graphically the student's time use summa- 
rized on worksheet 2. The specific time-on-task and off-task categories have been listed on the left , 
side of the sheet. Develop a bar graph by darkening the dotted line for each category to corre- 
spond with the percentage (measured on the horizontal axis). 

Task 
Categories 

Sanctioned 

• Setting Up 

• Cleaning Up - 

• Restroom Break 

• Scheduled Break • 

• Waiting for Directions 

Nonsanctioned 

• Doing Nothing 

• Non-Job-related 

Socializing - - - - 

• Disruptive Behavior - — — — 



Total Time off Task 



0 



10 



20 30 



40 



50 



60 



70 



80 



90 



100 



(Percentage of Time off Task) 
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WORKSHEET 5 
COOPERATIVE VOCATIONAL EDUCATOR'S ACTION PLAN 



DIRECTIONS: Use this worksheet to decide what changes you will make to enhance the coopera- 
tive education student's learning experience. Refer to worksheet 2 for information about the cur- 
rent use of time at the student's training station. In developing goals and strategies, consider 
school and work site policies and any other potential limitations. 



p 

f 

1 

0 

r 

1 
i 

y 


Observed use of 
co-op student time 


Goal for student 
use of time 


Specific strategies 
for reaching goal 


1 








2 








3 








4 








5 









Cooperative Education Coordinator 

Student 

Date 

Training Station 
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